Integrating boosting and stochastic attribute selection committees for further improving the performance of decision tree learning

Techniques for constructing classiier committees including Boosting and Bagging have demonstrated great success, especially Boosting for decision tree learning. This type of technique generates several classiiers to form a committee by repeated application of a single base learning algorithm. The committee members vote to decide the nal classiication. Boosting and Bagging create diierent classiiers by modifying the distribution of the training set. Sasc (Stochastic Attribute Selection Committees) uses an alternative approach to generating classiier committees by stochastic manipulation of the set of attributes considered at each node during tree induction, but keeping the distribution of the training set unchanged. In this paper, we propose a method for improving the performance of Boosting. This technique combines Boosting and Sasc. It builds classiier committees by manipulating both the distribution of the training set and the set of attributes available during induction. In the synergy, Sasc eeectively increases the model diversity of Boosting. Experiments with a representative collection of natural domains show that, on average, the combined technique outperforms both Boosting and Sasc in terms of reducing the error rate of decision tree learning.